Audio signal encoding method combining codes having different frame lengths and data rates

ABSTRACT

A system for transmitting audio signals over a telecommunications link generates the signals as two or more alternative feeds, for example at different data rates. The two feeds are encoded using coding methods having a frame structure with different frame lengths. To facilitate switching between the two, the input signal is notionally divided into temporal portions and each is coded by taking it, plus enough of the next (or preceding) portion to make up a whole number of frames, and encoding it, whereby the encoded portions overlap—at least for one of the feeds. The overlap is lost upon decoding by discarding duplicate material.

This application is the US national phase of international applicationPCT/GB01/05082 filed 19 Nov. 2001 which designated the U.S.

The present invention is concerned with the delivery, over atelecommunications link, of digitally coded material for presentation toa user.

According to one aspect of the invention there is provided a method ofencoding audio signals comprising

notionally dividing an input signal into successive temporal portions;

encoding said input temporal portions using a first encoding algorithmhaving a first frame length to produce a first encoded sequence ofencoded temporal portions;

encoding said input temporal portions using a second frame length toproduce a second sequence of encoded temporal portions;

wherein at least one of the encoding steps comprises encoding one inputtemporal portion along with so much of the end of the preceding temporalportion and/or the beginning of the immediately following temporalportion as to constitute with said one temporal portion an integralnumber of frames.

In another aspect, the invention provides a method of encoding inputaudio signals comprising: encoding with a first coding algorithm havinga first frame length each of successive first temporal portions of theinput signal, which portions correspond to an integral number of saidfirst frame lengths and either are contiguous or overlap, to produce afirst encoded sequence; encoding with a second coding algorithm having asecond frame length each of successive second temporal portions of theinput signal, which portions correspond to an integral number of saidsecond frame lengths and do not correspond to an integral number of saidfirst frame lengths and which overlap, to produce a second encodedsequence such that each overlap region of the second encoded sequenceencompasses at least partially a boundary between, or, as the case maybe, overlap region portions of, the first encoded sequence whichcorrespond to successive temporal portions of the input signal.

Other, optional, aspects of the invention are set out in the sub-claims.

Note that the following description and drawings is identical to thatcontained in our co-pending international patent application entitled“Delivery of Audio and or Video Material” (Applicant's ref A26097) filedon the same day as the present application, claiming priority from GB 0030706.6.

Some embodiments of the present invention will now be described, withreference to the accompanying drawings, in which:

FIG. 1 is a diagram illustrating the overall architecture of the systemsto be described;

FIG. 2 is a block diagram of a terminal for use in such a system;

FIG. 3 shows the contents of a typical index file;

FIG. 4 is a timing diagram illustrating a modified method of sub-filegeneration; and

FIG. 5 is a diagram illustrating a modified architecture.

The system shown in FIG. 1 has as its object, the delivery, to a user,of digitally coded audio signals (for example, of recorded music orspeech) via a telecommunications network to a user terminal where thecorresponding sounds are to be played to the user. However, as will bediscussed in more detail below, the system may be used to convey videosignals instead of, or in addition to, audio signals. In this example,the network is the Internet or other packet network operating inaccordance with the Hypertext Transfer Protocol (see RFCs 1945/2068 fordetails), though in principle other digital links or networks can beused. It is also assumed that the audio signals have been recorded incompressed form using the ISO MPEG-1 Layer III standard (the “MP3standard”); however it is not essential to use this particular format.Nor, indeed, is it necessary that compression be used, though naturallyit is highly desirable, especially if the available bit-rate isrestricted or storage space is limited. In FIG. 1, a server 1 isconnected via the internet 2 to user terminals 3, only one of which isshown. The function of the server 1 is to store data files, to receivefrom a user terminal a request for delivery of a desired data file and,in response to such a request, to transmit the file to the user terminalvia the network. Usually such a request takes the form of first partindicating the network delivery mechanism (e.g. http:// or file:// forthe hypertext transfer protocol or file transfer protocol respectively)followed by the network address of the server (e.g. www.server 1.com)suffixed with the name of the file that is being requested. Note that,in the examples given, such names are, for typographical reasons, shownwith the “//” replaced by “\\”.

In these examples, the use of the hypertext transfer protocol isassumed; this is not essential, but is beneficial in allowing use of theauthentication and security features (such as the Secure Sockets Layer)provided by that protocol.

Conventionally, a server for delivery of MP3 files takes the form of aso-called streamer which includes processing arrangements for thedynamic control of the rate at which data are transmitted depending onthe replay requirements at the user terminal, for the masking of errorsdue to packet loss and, if user interaction is allowed, the control ofthe flow of data between server and client; here however the server 1contains no such provision. Thus it is merely an ordinary “web server”.

The manner in which the data files are stored on the server 1 will nowbe explained. Suppose that an MP3-format file has been created and is tobe stored on the server. Suppose that it is a recording of J. S. Bach'sToccata and Fugue in D minor (BWV565) which typically has a playing timeof 9 minutes. Originally this would have been created as a single datafile, and on a conventional streamer would be stored as this one singlefile. Here, however, the file is divided into smaller files before beingstored on the server 1. We prefer that each of these smaller files is ofa size corresponding to a fixed playing time, perhaps four seconds. Witha compressed format such as MP3 this may mean that the files will be ofdifferent sizes in terms of the number of bits they actually contain.Thus the Bach file of 9 minutes duration would be divided into 135smaller files each representing four seconds' playing time. In thisexample these are given file names which include a serial numberindicative of their sequence in the original file, for example:

-   -   000000.bin    -   000001.bin    -   000002.bin    -   000003.bin    -   . . .    -   000134.bin

The partitioning of the file into these smaller sub-files may typicallybe performed by the person preparing the file for loading onto the webserver 1. (The expression “sub-files” is used here to distinguish themfrom the original file containing the whole recording: it should howeverbe emphasised that, as far as the server is concerned each “sub-file” isjust a file like any other file). The precise manner of their creationwill be described more fully below. Once created, these sub-files areuploaded onto the server in a conventional manner just like any otherfile being loaded onto a web server. Of course the filename could alsocontain characters identifying the particular recording (the sub-filecould also be “tagged” with additional information—when you play an MP3file you get information on the author, copyright etc), but in thisexample the sub-files are stored on the server in a directory or folderspecific to the particular recording—e.g. mp3_bwv565. Thus a sub-file,when required, may be requested in the form:

-   -   http:\\www.server1.com/mp3_bwv565/000003.bin

where “www.server1.com” is the URL of the server 1.

It is also convenient for the person preparing the sub-files for loadingonto the server to create, for each recording, a link page (typically inhtml format) which is also stored on the server (perhaps with filenamemp3_bwv565/link.htm), the structure and purpose of which will bedescribed later.

It is also convenient that the web server stores one or more (html) menupages (e.g. menu.htm) containing a list of recordings available, withhyperlinks to the corresponding link pages.

Turning now to the terminal, this may typically take the form of aconventional desktop computer, with, however, additional software forhandling the reception of the audio files discussed. If desired, theterminal could take the form of a handheld computer, or even beincorporated into a mobile telephone. Thus FIG. 2 shows such a terminalwith a central processor 30, memory 31, a disk store 32, a keyboard 33video display 34, communications interface 35, and audio interface(“sound card”) 36. For video delivery, a video card would be fitted inplace of, or in addition to, the card 36. In the disk store are programswhich may be retrieved into the memory 31 for execution by the processor30, in the usual manner. These programs include a communications program37 for call-up and display of html pages—that is, a “web browser”program such as Netscape Navigator or Microsoft Explorer, and a furtherprogram 38 which will be referred to here as “the player program” whichprovides the functionality necessary for the playing of audio files inaccordance with this embodiment of the invention. Also shown is a region39 of the memory 31 which is allocated as a buffer. This is a decodedaudio buffer containing data waiting to be played (typically the playouttime of the buffer might be 10 seconds). The audio interface or soundcard 36 can be a conventional card and simply serves to receive PCMaudio and convert it into an analogue audio signal, e.g. for playingthrough a loudspeaker. Firstly, we will give a brief overview of theoperation of the terminal for the retrieval and playing of the desiredrecording when using the HTTP and an embedded or “plugin” client

-   1. The user uses the browser to retrieve and display the menu page    menu.htm from the server 1.-   2. The user selects one of the hyperlinks within the menu page which    causes the browser to retrieve from the server, and display, the    link page for the desired recording—in this example the file    mp3_bwv565_link.htm. The actual display of this page is unimportant    (except that it may perhaps contain a message to reassure the user    that the system is working correctly). What is important about this    page is that it contains a command (or “embed tag”) to invoke in the    processor 30 a secondary process in which the player program 37 is    executed. The invocation of a secondary process in this manner is    well-known practice (such a process is known in Netscape systems as    a “plug-in” and in Microsoft systems as “ActiveX”). Such commands    can also contains parameters to be passed to the secondary process    and in the system of FIG. 1 the command contains the server URL of    the recording, which, for the Bach piece, would be    http:\\www.server1/mp3_bwv565.-   3. The player program 37 includes an MP3 decoder, the operation of    which is, in itself, conventional. Of more interest in the present    context are the control functions of the program which are as    follows.-   4. The player program, having received the URL, adds to this the    filename of the first sub-file, to produce a complete address for    the sub-file—i.e. www.server1/mp3_bwv565/000000.bin. It will be    observed that this system is organised on the basis that the    sub-files are named in the manner indicated above, so that the    terminal does not need to be informed of the filenames. The program    constructs a request message for the file having this URL and    transmits it to the server 1 via the communications interface 35 and    the internet 2. (Processes for translating the URL into an IP    address and for error reporting of invalid, incomplete or    unavailable URLs are conventional and will not therefore be    described). We envisage that the player program would send the    requests directly to the communications interface, rather than via    the browser. The server responds by transmitting the required    sub-file.-   5. The player program determines from the file the audio decoding    used in this sub-file and decodes the file back to raw PCM values in    accordance with the relevant standard (MP3 in this example), making    a note of the play time of this sub-file. Generally an audio file    contains an identifier at the beginning of the file which states the    encoding used. The decoded audio data is then stored in the audio    buffer 38.-   6. The player program has a parameter called the playout time T_(p).    In this example it is set at 10 seconds (it could be made    user-selectable, if desired). It determines the degree of buffering    that the terminal performs.-   7. The player program increments the filename to 000001.bin and    requests, receives, decodes and stores this second sub-file as    described in (4) and (5) above. It repeats this process until the    contents of the buffer reach or exceed the playout time T_(p). Note    that it is not actually essential that the decoding occurs before    the buffer but it simplifies matters as since the audio is decoded    back to raw PCM then the duration of the buffered material is then    explicitly known. It simplifies the control of the audio buffer if    each of the sub-files is the same audio playback size.-   8. Having reached the playout threshold T_(p) the decoded data are    sent from the buffer to the audio interface 36 which plays the sound    through a loudspeaker (not shown).-   9. Whilst playing the sounds as in (8) above, the player program    continually monitors the state of buffer fullness and whenever this    falls below T_(p) it increments the filename again and obtains a    further sub-file from the server. This process is repeated until a    “file not found error” is returned.-   10. If, during this process, the buffer becomes empty, the player    program simply ceases playing until further data arrives.

The sub-file naming convention used here, of a simple fixed lengthsequence of numbers starting with zero, is preferred as it is simple toimplement, but any naming convention can be used provided the playerprogram either contains (or is sent) the name of the first sub-file andan algorithm enabling it to calculate succeeding ones, or alternativelyis sent a list of the filenames.

It will have been observed that the system described above offers theuser no opportunity to intervene in the replay process. Nor does itoffer any remedy for the possibility of buffer underflow (due forexample to network congestion). Therefore a second, more sophisticatedembodiment of the invention, now to be described, offers the followingfurther features:

-   a) the server stores two or more versions of the recording, recorded    at different compression rates (for example at compressions    corresponding to (continuous) data rates of 8, 16, 24 and 32 kbit/s    respectively) and the player program is able to switch automatically    between them.-   b) the player program displays to the user a control panel whereby    the user may start the playing, pause it, restart it (from the    beginning, or from the point at which it paused), or jump to a    different point in the recording (back or forward).

Note that these features are not interdependent, in that user controlcould be provided without rate-switching, or vice versa.

In order to provide for rate switching, the person preparing the filefor loading onto the server prepares several source files—by encodingthe same PCM file several times at different rates. He then partitionseach source file into sub-files, as before. These can be loaded onto theserver in separate directories corresponding to the different rate, asin the following example structure, where “008k”, “024k” in thedirectory name indicates a rate of 8 kbit/s or 24 kbit/s and so on.

He also creates an index file (e.g. index.htm) the primary purpose ofwhich is to provide a list of the data rates that are available.

Directory Subdirectory Filename mp3_bwv565 none link.htm index.htmmp3_bwv565 008k_11_m 000000.bin 000001.bin 000002.bin 000003.bin . . .000134.bin mp3_bwv565 016k_11_m 000000.bin 000001.bin 000002.bin000003.bin . . . 000134.bin mp3_bwv565 018k_11_s 000000.bin 000001.bin000002.bin 000003.bin . . . 000134.bin mp3_bwv565 024k_11_s 000000.bin000001.bin 000002.bin 000003.bin . . . 000134.bin mp3_bwv565 032k_11_s000000.bin 000001.bin 000002.bin 000003.bin . . . 000134.bin Note thatbecause the length of a sub-file corresponds, as explained earlier, to afixed length of time, the number of sub-files is the same for eachdirectory. The subdirectory names comprise the data rate in kbit/s(three digits) plus the letter “k”; in this example indications of theaudio sampling rate (11.025 kHz) and a mono-stereo flag are appended,for verification purposes.

The index file would thus contain a statement of the form:<!Audio=“024k_(—)11_s 032k_(—)11_s 018k_(—)11_s 016k_(—)11_m008_(—)11_m”-->

(The <!--...--> simply indicates that the statement is embedded as acomment in an html file (or a simple text file could be used)). Atypical index file is shown in FIG. 3 where other information isincluded: LFI is the highest sub-file number (i.e. there are 45sub-files) and SL is the total playing time (178 seconds). “Mode”indicates “recorded” (as here) or “live” (to be discussed below). Theother entries are either self-explanatory, or standard html commands.

Initially the player program will begin by requesting, from thedirectory specified in the link file, the index file, and stores locallya list of available data rates for future reference. (It may explicitlyrequest this file or just specify the directory: most servers default toindex.htm if a filename is not specified.) It then begins to request theaudio sub-files as described earlier, from the first-mentioned “rate”directory in the index file—viz. 024k_(—)11_s (or the terminal couldoverride this by modifying this to a default rate set locally for thatterminal). The process from then on is that the player program measuresthe actual data rate being received from the server, averaged over aperiod of time (for example 30 seconds). It does this by timing everyURL request; the transfer rate achieved (number of bits per second)between the client and server is determined. The accuracy of this figureimproves as the number of requests goes up. The player maintains twostored parameters which indicate, respectively, the current rate, andthe measured rate.

The initiation of a rate change is triggered:

-   a) if the buffer ever empties AND the measured rate is less than the    current rate AND the measured Buffer Low Percentage exceeds a Step    Down Threshold (as described below), reduce the current rate;    (changing at a time when the buffer is already empty is advantageous    as the sound card is not playing anything and it may be necessary to    reconfigure it if the audio sampling rate, stereo-mono setting or    bit width (number of bits per sample) has changed).-   b) if the measured rate exceeds not only the current rate but also    the next higher rate for a given period of time (e.g. 120 seconds:    this could if desired be made adjustable by the user) increase the    current rate

The Buffer Low Percentage is the percentage of the time that the buffercontents represent less than 25% of the playout time (i.e. the buffer isgetting close to being empty). If the Step Down Threshold is set to 0%then when the buffer empties the system always steps down when the otherconditions are satisfied. Setting the Step Down Threshold to 5% (this isour preferred default value) means that if the buffer empties but themeasured Buffer Low Percentage is greater than 5% it will not step down.Further buffer empties will obviously cause this measured rate toincrease and will eventually empty the buffer again with a Buffer LowPercentage value exceeding 5% if the rate can not be sustained. Settingthe value to 100% means the client will never step down.

The actual rate change is effected simply by the player program changingthe relevant part of the sub-file address for example, changing “008k”to “024k” to increase the data rate from 8 to 24 kbit/s, and changingthe current rate parameter to match. As a result, the next request tothe server becomes a request for the higher (or lower) rate, and thesub-file from the new directory is received, decoded and entered intothe buffer. The process just described is summarised in the followingflowchart:

User Terminal Server Select Menu page Requesthttp:\\server1.com/menu.htm Send http:\\server1.com/menu.htm Displaymenu.htm Select item from Menu (Bach) Extract hyperlink URL frommenu.htm (mp3__bwv565/link.htm) Requesthttp:\\server1.com/mp3_bwv565/link.htm Send http:\\server1.com/mp3_bwv565/link.htm Display link.htm Execute secondary process (playerprogram) specified in link.htm with parameters specified in link.htm(http:\\server1/mp3_bwv565) Set Stem to that specified Set URL = Stem +“index.htm” Request this URL Send requested file Set Rate List to ratesspecified in index.htm Set LFI to value specified in index.htm Set StemC= Stem + “/” + RateList(item 1) Set CurrentRate = rate specified inRateList (item 1) Set RateU = next higher rate in list or zero if noneSet StemU = Stem + “/” + item in rate list corresponding to this rate;Set RateD = next lower rate in list or zero if none Set StemD = Stem +“/” + item in rate list corresponding to this rate; Set Current Subfile= 000000.bin J1: Set URL = StemC + Current Subfile Request this URL Ifrequested subfile exists, Send requested subfile; otherwise send errormessage If error message received, Stop Decode received subfile Write tobuffer JIA: If Buffer Fullness > Tp seconds go to Step J3 J2: IncrementCurrent Subfile Go to Step J1 J3: Begin/continue playing of buffercontents via sound card J4: If Buffer Fullness < Tp seconds go to StepJ2 If BufferFullness = 0 AND Mrate < CurrentRate AND BufferLow% > Td goto Stepdown If MRate > NextRate AND NextRate <> 0 goto Stepup Input ofuser If UserCommand = Pause then: commands Stop reading from buffer;Loop until Usercommand = Resume; go to J3 If UserCommand = Jump(j%)then: Clear buffer;  Set CurrentSubfile =  Integer[(LFI + 1) * j / 100];  goto Step J1 Go to Step J1A Stepup: Clear Buffer Set RateD = RateC SetStemD = StemC Set RateC = RateU Set StemC = StemU Set RateU = nexthigher rate in list or zero if none Set StemU = Stem + “/” + item inrate list corresponding to this rate Go to Step J1A Stepdown: ClearBuffer Set RateU = RateC Set StemU = StemC Set RateC = RateD Set StemC =StemD Set RateD = next lowerr rate in list or zero if none Set StemD =Stem + “/” + item in rate list corresponding to this rate Go to Step J1A

The user control is implemented by the user being offered on the screenthe following options which he can select using the keyboard or otherinput device such as a mouse:

-   a) Start: implement the numbered steps given above, from step 4.    Whether, when a recording is first selected, it begins to play    automatically, or requires a Start instruction from the user, is    optional; indeed, if desired, the choice may be made by means of an    additional “autoplay” parameter in the link file.-   b) Pause: implemented by an instruction to the MP3 decoder to    suspend reading data from the buffer;-   c) Resume: implemented by an instruction to the MP3 decoder to    resume reading data from the buffer;-   d) Jump: implemented by the user indicating which part of the    recording he wishes to jump to—for example by moving a cursor to a    desired point on a displayed bar representing the total duration of    the recording; the player then determines that this point is x %    along the bar and calculates the number of the next sub-file needed,    which is then used for the next request. In the Bach example with    125 sub-files then a request to play from a point 20% into the    recording would result in a request for the 26th sub-file—i.e.    000025.bin. It will be apparent that this calculation is    considerably simplified if each sub-file corresponds to the same    fixed duration. We prefer, in the case of the jump, to suspend    decoding and clear the buffer so that the new request is sent    immediately, but this is not actually essential.

It is of interest to discuss further the process of partitioning theoriginal file into sub-files. First, it should be noted that if (as inthe first version described above), there is no expectation that asub-file will be followed by a sub-file other than that whichimmediately follows it in the original sequence, then it matters littlewhere the boundaries between the sub-files are located. In that case thesub-file size can be a fixed number of bits, or a fixed playing timelength (or neither of these)—the only real decision is how big thesub-files should be. Where jumps are envisaged (in time, or betweendifferent data rates) there are other considerations. Where, as withmany types of speech or audio coding (including MP3), the signal iscoded in frames, a sub-file should contain a whole number of frames. Inthe case of rate switching, it is, if not actually essential, highlydesirable that the sub-file boundaries are the same for each rate, sothat the first sub-file received for a new rate continues from the samepoint in the recording that the last sub-file at the old rate ended. Toarrange that every sub-file should represent the same fixed time period(e.g. the 4 seconds mentioned above) is not the only way of achievingthis, but it is certainly the most convenient. Note however that,depending on the coding system in use, the requirement that a sub-fileshould contain a whole number of frames may mean that the playingduration of the sub-files does vary slightly. Note that in thisembodiment of the invention, the available data rates, though they usedifferent degrees of quantisation, and differ as to whether they encodein mono or stereo, all use the same audio sampling rate and inconsequence the same frame size. Issues that need to be addressed whendiffering frame sizes are used are discussed below.

As for the actual sub-file length, excessively short sub-files shouldpreferably be avoided because (a) they create extra network traffic inthe form of more requests, and (b) on certain types of packetnetworks—including IP networks—they are wasteful in that they have to beconveyed by smaller packets so that overhead represented by therequesting process and the packet header is proportionately greater. Onthe other hand, excessively large sub-files are disadvantageous inrequiring a larger buffer and in causing extra delay a when startingplay and/or when jumps or rate changes are invoked. A sub-file size ofbetween 30% and 130% of the playout time, or preferably around half theplayout time (as in the examples given above), is found to besatisfactory.

The actual process of converting the sub-files can be implemented bymeans of a computer programmed in accordance with the criteriadiscussed. Probably it will be convenient to do this on a separatecomputer, from which the sub-files can be uploaded to the server.

Another refinement that can be added is to substitute a more complexsub-file naming convention so as to increase security by making it moredifficult for an unauthorised person to copy the sub-files and offerthem on another server. One example is to generate the filenames using apseudo-random sequence generator, e.g. producing filenames of the form:

01302546134643677534543134.bin

94543452345434533452134565.bin

. . .

In this case the player program would include an identical pseudo-randomsequence generator. The server sends the first filename, or a “seed” ofperhaps four digits, and the generator in the player can thensynchronise its generator and generate the required sub-file names inthe correct sequence.

In the above example of rate-switching, all the data rates used had thesame frame size, specifically they used MP3 coding of PCM audio sampledat 11.025 KHz and a (PCM) frame size of 1152 samples. If it is desiredto accomplish rate switching between MP3 (or other) recordings havingdifferent frame sizes, problems arise due to the requirement that asub-file should contain a whole number of frames, because the frameboundaries do not then coincide. This problem can be solved by thefollowing modified procedure for creating the sub-files. It should benoted particularly that this procedure can be used in any situationwhere rate switching is required and is not limited to the particularmethod of delivery discussed above.

FIG. 4 shows diagrammatically a sequence of audio samples, upon whichsuccessive four-second segments are delineated by boundary marks (in thefigure) B1, B2 etc. At 11.025 KHz, there are 44,100 samples in eachsegment.

-   1. Encode the audio, starting at boundary B1, frame by frame, to    create an MP3 sub-file, continuing until a whole number of frames    having a total duration of at least four seconds has been encoded.    With a frame size of 1152 samples, four seconds corresponds to 38.3    frames, so a sub-file S1 representing 39 frames will actually be    encoded, representing a total duration of 4.075 seconds.-   2. Encode the audio, in the same manner, starting at boundary B2.-   3. Repeat, starting each time at a 4-second boundary, so that in    this way a set of overlapping sub-files is generated for the whole    audio sequence to be coded. The last segment (which may well be    shorter than four seconds) has of course nothing following it, and    is padded with zeroes (i.e. silence).

Coding of the other data rates using different frame sizes proceeds inthe same manner.

At the terminal, the control mechanisms are unchanged, but the decodingand buffering process is modified:

-   1. Receive sub-file S1;-   2. Decode sub-file S1;-   3. Write into the buffer only the first four seconds of the decoded    audio samples (discard the remainder);-   4. Receive sub-file S2;-   5. Decode sub-file S2;-   6. Write into the buffer only the first four seconds of the decoded    audio samples;-   7. Continue with sub-file S3 etc.

In this way, it is ensured that the sub-file sets for all rates havesub-file boundaries which correspond at the same points in the originalPCM sample sequence.

Thus, each four-second period except the last is, prior to encoding,“padded” with audio samples from the next four-second period so as tobring the sub-file size up to a whole number of MP3 frames. If desired,the padding samples could be taken from the end of the precedingfour-second period instead of (or as well as) the beginning of thefollowing one.

Note that the MP3 standard allows (by a scheme known as “bit reservoir”)certain information to be carried over from one audio frame to another.In the present context, while this is acceptable within a sub-file, itis not acceptable between sub-files. However, since naturally thestandard does not allow such carry-over at the end or beginning of arecording, this problem is easily solved by encoding each sub-fileseparately, as if it were a single recording.

Changes of sampling rate (and indeed switching between mono and stereooperation) have some practical implications for operation of the audiointerface 36. Many conventional sound cards, although capable ofoperation at a range of different settings, require re-setting in orderto change sampling rate, and necessarily this causes an interruption inits audio output. Thus in a further modification, we propose that thesound card could be run continuously at the highest sampling rateenvisaged. When the player program is supplying, to the buffer, data ata lower sampling rate, this data is then up-sampled to this highest ratebefore or after the buffer. Similarly, if the card is always operated instereo mode, decoded mono signals can be fed in parallel so feed boththe left and right channels of the sound card input. Again, if thenumber of bits per sample of the decoded signal is lower than expectedby the card, the number of bits can be increased by padding with zeros.

Recollecting that the criteria discussed earlier for automatic data rateswitching downwards envisaged a rate reduction only in cases of bufferunderflow (involving therefore interruptions in the output), we notethat with this modification such interruption can be avoided andtherefore a criterion which anticipates underflow and avoids it in themajority of cases. In this case the first of the three AND conditionsmentioned above (namely, that the buffer is empty) would be omitted.

The same principle may be applied to the delivery of video recordings,or of course, video recordings with an accompanying sound track. In thesimpler version, where there is only one recording, the system differsfrom the audio version only in that the file is a video file (e.g. inH.261 or MPEG format) and the player program incorporates a videodecoder. The manner of partitioning the file into sub-files isunchanged.

As in the audio case, there may be two or more recordings correspondingto different data rates, selected by the control mechanism alreadydescribed. Also one can provide additional recordings corresponding todifferent replay modes such as fast forward or fast reverse which can beselected by an extension of the user control facilities alreadydescribed. Again, a systematic convention for file and directory namingcan be followed so that the player program can respond to—for example—afast forward command by amending the sub-file address.

The delivery of video recordings does however have further implicationsfor file partitioning if switching or jumps are to be permitted. In thecase of recordings where each frame of a picture is coded independently,it is sufficient that a sub-file contains a whole number of frames of apicture. If compression involving inter-frame techniques is in use,however, the situation is more complex. Some such systems (for examplethe MPEG standards) generate a mixture of independently coded frames(“intra-frames”) and predictively coded frames; in this case eachsub-file should preferably begin with an intra-frame.

In the case of inter-frame coding systems such as the ITU H.261standard, which do not provide for the frequent, regular inclusion ofintra-frames, this is not possible. This is because—takingrate-switching as an example, if one were to request sub-file n of ahigher bit rate recording followed by sub-file n+1 of a lower bit-raterecording, the first frame of the lower bit-rate sub-file would havebeen coded on an inter-frame basis using the last decoded frame ofsub-file n of the lower rate recording, which of course the terminaldoes not have at its disposal—it has the last decoded frame of sub-filen of the higher rate recording. Thus serious mistracking of the decoderwould occur.

In the case of switching between normal play and a fast play mode, thesituation is in practice slightly different. On fast forward play at,for example, 5 times normal speed, one encodes only every 5th frame. Inconsequence the inter-frame correlation is much reduced and inter-framecoding becomes unattractive, so one would generally prefer to encode afast play sequence as intra-frames. Switching from normal to fast thenpresents no problem, as the intra-frames can be decoded withoutdifficulty. However, when reverting to normal play, the mistrackingproblem again occurs because the terminal is then presented with apredictively coded frame for which it does not have the preceding frame.

In either case the problem can be solved by using the principledescribed in our international patent application No. WO98/26604 (issuedin USA as U.S. Pat. No. 6,002,440). This involves the encoding of anintermediate sequence of frames which bridges the gap between the lastframe of the preceding sequence and the first frame of the new sequence.

The operation of this will now be described in the context of fastforward operation (fast rewind being similar but in reverse). In thisexample we assume that a 9 minute video sequence has been encoded at 96kbit/s according to the H.261 standard, and again at 5 times normal rateentirely at H.261 infra-frames, and that the resulting files have eachbeen partitioned into four-second sub-files. Here, four seconds refersto the duration of the original video signal, not to the fast forwardplaying time. Following a naming convention similar to that employedabove, the sub-files might be:

Directory Subdirectory Filename mpg_name 096k_x1 000000.bin 000001.bin .. . 000134.bin 096k_x5 000000.bin . . . 000134.bin where “name” is aname to identify the particular recording, “x1” indicates normal rateand “x5” indicates five times normal rate - i.e. fast forward.

To switch from normal play to fast forward is only necessary for theplayer program to modify the sub-file address to point to the fastforward sequence—e.g.

Request mpg_name/096k_(—)×1/000055.bin is followed by

Request mpg_name/096K_(—)×5/000056.bin

In order to construct the bridging sequences for switching back tonormal play it is necessary to construct a bridging sub-file for eachpossible transition. As described in our international patentapplication mentioned above, a sequence of three or four frames isgenerally sufficient for bridging, so a simple method of implementationis to construct bridging sub-files of only 4 frames duration—e.g.

Directory Subdirectory Filename mpg_name 096K_5 > 1 0000001.bin . . .000133.bin

So that the switching is accomplished by a series of requests such as:

Request mpg_name/096k_(—)×5/000099.bin

Request mpg_name/096k_(—)>1/000099.bin

Request mpg_name/096k_(—)×1/000100.bin

The bridging sub-file is generated as follows:

-   -   Decode the fast forward sequence to obtain a decoded version of        the last frame of sub-file 99, (at 25 frames per second this        will be frame 100,000 of the original video signal).    -   Decode the normal sequence to obtain a decoded version of the        first frame of sub-file 100 (i.e. frame 100,001). Re-encode this        one frame four times using H.261 inter-frame coding based on the        decoded frame 100,000 as the initial reference frame.    -   Thus, when the decoder has decoded the fast forward sub-file,        followed by the bridging sub-file it will have reconstructed        frame 100,000 correctly and will be ready to decode the normal        (×1) frames. Incidentally, the reason that one encodes the same        frame several times in this procedure is that doing so merely        once, would produce poor picture quality due to the quantisation        characteristics of H.261.

Exactly the same process could be used for rate-switching (albeit thatnow bridging sub-files are required in both directions). However, itwill be observed that, as described, the bridging sub-file results in afreezing of the picture for a period of four frames—i.e. (at 25 framesper second) 160 ms. In switching from fast to normal play this isacceptable—indeed one would probably choose to clear the buffer at thispoint. It may or may not be subjectively acceptable on rate-switching.An alternative therefore would be to construct a four-second bridgingsequence. The request series would then look like:

mpg_name/096k_(—)×1/000099.bin

mpg_name/096/128_(—)×1/000100.bin

mpg_name/128k_(—)×1/000101.bin

The bridging sub-file would in that case be constructed either byrecoding the fifth decoded frame of the decoded 128 kbit/s sequence fourtimes starting with decoded 96 kbit/sframe 100,000 as the referenceframe, or coding the first four frames of the decoded 128 kbit/ssequence starting with decoded 96 kbit/s frame 100,000 as the referenceframe. In both cases the remaining 96 frames of the bridging sub-filewould be a copy of the 128 kbit/s sub-file.

The files to be delivered have been referred to as “recordings”.However, it is not necessary that the entire audio or video sequenceshould have been encoded—or even exist—before delivery is commenced.Thus a computer could be provided to receive a live feed, to code itusing the chosen coding scheme, and generate the sub-files “on the fly”and upload them to the server, so that, once a few sub-files are presenton the server, delivery may commence.

One application of this delivery system would be for a voice-messagingsystem, as illustrated in FIG. 5 where the server 1, network 2 andterminal 3 are again shown. A voice-messaging interface 4 serves toreceive telephone calls, for example via the public switched telephonenetwork (PSTN) 5, to record a message, encode it, partition it intosub-files, and upload them to the server 1, where they can be accessedin the manner described earlier. Alternatively a second interface 6could be provided, operating in a similar manner to the terminal 3 butcontrolled remotely via the PSTN by a remote telephone 5, to which thereplayed audio signals are then sent.

The same system can be used for a live audio (or video) feed. It is in asense still “recorded”—the difference being primarily that delivery andreplay commence before recording has finished, although naturally thereis an inherent delay in that one must wait until at least one sub-filehas been recorded and loaded onto the server 1.

The system can proceed as described above, and would be quitesatisfactory except for the fact that replay would start at thebeginning whereas what the user will most probably want is for it tostart now—i.e. with the most recently created sub-file.

With a lengthy audio sequence one may choose to delete the oldersub-files to save on storage: with a continuous feed (i.e. 24 hours aday) this will be inevitable and moreover one would need to reuse thesub-file names (in our prototype system we use 000000.bin to 009768.binand then start again at 000000.bin), so that the older sub-files areconstantly overwritten with the new ones. A simple method of ensuringdelivery starting with the most recent sub-file would be to include inthe index file an extra command instructing the player program to startby requesting the appropriate sub-file. This however has thedisadvantage that the index file has to be modified veryfrequently—ideally every time a new sub-file is created. Therefore wepropose a method whereby the player program scans the server to find thestarting sub-file, as follows. In the index file, the Mode parameter isset to “live” to trigger the player program to invoke this method. LFIis set to indicate the maximum number of sub-files that may bestored—say 9768. The method involves the following steps and presupposesthat (as is conventional) each sub-file's “last modified” time and datehas been determined. When using the HTTP protocol this can be achievedusing a HEAD request which results not in delivery of the requestedsub-file but only of header information indicating the time that thesub-file was written to the server, or zero if the sub-file does notexist. This time is represented below as GetURL(LiveIndex) whereLiveIndex is the sequence number of the sub-file in question. Commentsare preceded by “//”.

1 LFI = 9768 // read from the index.htm file LiveIndex = LFI / 2StepSize = LFI / 2 LiveIndexModifiedAt = 0; // the beginning of time. 10ThisIndexWasModifiedAt = GetURL(LiveIndex); 20 If (StepSize = 1) [was If(StepSize == 1)] { // LiveIndexModifiedAt contains the time the file waswritten or 0 if no file // has been found. LiveIndex contains the index.goto 30 [was FINISH] } StepSize = StepSize / 2 if(ThisIndexWasModifiedAt > LiveIndexModifiedAt) { LiveIndexModifiedAt =ThisIndexesModifiedAt; LiveIndex = LiveIndex + StepSize } else {LiveIndex = LiveIndex − StepSize } Goto 10 30 FINISH

Having found the LiveIndex it is prudent to step back the Tp (playouttime) and start to make the requests to fill the audio buffer fromthere. Playing may commence in the normal way.

Once the recording has actually finished, the index file can if desiredbe modified to set Mode to “recorded”, and any length parameters.

If desired the player program could check periodically to see whetherthe index file has changed from “live” to “recorded” mode and if so toswitch to “recorded” mode playing.

A simpler and much faster method of the identification of the “latest”sub-file will now be described, assuming, first of all, a singlecontinuous sub-file numbering sequence.

1. Terminal issues a HEAD request for the first sub-file (e.g.000000.bin).

2. The server replies by sending the header of this file and includesthe date and time the file was last modified (MODTIME) and the date andtime at which this reply was sent (REPLYTIME) (both of these arestandard http. fields).

3. The terminal calculates the elapsed time (ELTIME) by subtracting thetwo (ELTIME=REPLYTIME−MODTIME), and divides this by the playing durationof a sub-file (4 seconds, in these examples) to obtainLIVEINDEX=ELTIME/4.

4. The terminal calculates the filename of the sub-file having thisindex.

5. The terminal issues a HEAD request with this filename and ifnecessary each subsequent filename until it receives zero (file notfound) whereupon it regards the latest sub-file which is found as the“Current sub-file”.

6. The terminal begins requesting files, starting at point J1: of theflowchart given earlier.

This method is considerably faster than that described above for thecyclically numbered sub-files. Note that older sub-files may still bedeleted, to reduce storage requirement, as long as the starting sub-fileis kept. The method can however be modified to accommodate filenamere-use (cyclic addresses), but would require:

(i) That the starting sub-file name (e.g. 000000.bin) is not re-used sothat it is always available to supply the header information at Step 2.Thus, with wrapping at 009768.bin, sub-file 009768.bin would be followedby sub-file 000001.bin.

(ii) The calculated LIVEINDEX at Step 3 is taken Modulo 9768 (i.e. theremainder when ELTIME/4 is divided by 9768).

(iii) Sub-file deletion always leads the creation of new sub-files sothat a few file-names between the newest sub-file and the oldestundeleted sub-file do not exist, in order that the expected “file notfound” response occurs at Step 5.

There may be a danger of the playing operation running slightly fasteror slower than the recording operation. To guard against the former itmay be arranged that the player program checks each sub-file it receivesto ascertain whether it is marked with a later time than the previousone: if not the sub-file is discarded and repeated requests made perhapsthree times) followed by a check of the index file if these requests areunsuccessful.

If the playing lags behind the recording process this can be identifiedby the player program occasionally checking the server for the existenceof a significant number of sub-files more recent than those currentlybeing requested, and if such sub-file do exist, initiating a “catchingup” process—e.g. by regularly discarding a small amount of data.

1. A method of encoding audio signals comprising notionally dividing aninput signal into successive temporal portions; encoding said inputtemporal portions using a first encoding algorithm having a first framelength to produce a first encoded sequence of encoded temporal portions;encoding said input temporal portions using a second frame length toproduce a second sequence of encoded temporal portions; wherein at feastone of the encoding steps comprises encoding one input temporal portionalong with so much of the end of the preceding temporal portion and/orthe beginning of the immediately following temporal portion as toconstitute with said one temporal portion an integral number of frames.2. A method according to claim 1 in which the first and second encodingalgorithms correspond to respective different output data rates.
 3. Amethod according to claim 1 including feeding one sequence of encodedtemporal portions to an input and, in response to a switching command,switching the output to be supplied with the other sequence of encodedtemporal portions, the switching occurring at the boundary between oneencoded temporat portion and the next.
 4. A method of transmitting audiosignals comprising: encoding the signals using the method of claim 1;decoding the discrete portions; and discarding that part of the decodedsignal which corresponds to said end and/or beginning.
 5. A method ofencoding input audio signals comprising: encoding with a first codingalgorithm having a first frame length each of successive first temporalportions of the input signal, which portions correspond to an integralnumber of said first frame lengths and either are contiguous or overlap,to produce a first encoded sequence; encoding with a second codingalgorithm having a second frame length each of successive secondtemporal portions of the input signal, which portions correspond to anintegral number of said second frame lengths and do not correspond to anintegral number of said first frame lengths and which overlap, toproduce a second encoded sequence such that each overlap region of thesecond encoded sequence encompasses at least partially a boundarybetween, or, as the case may be, overlap region portions of the firstencoded sequence which correspond to successive temporal portions of theinput signal.